1. Field of the Invention
The present invention relates to a method, system, and program for managing memory for data transmission through a network.
2. Description of the Related Art
In a network environment, a network adaptor on a host computer, such as an Ethernet controller, Fibre Channel controller, etc., will receive Input/Output (I/O) requests or responses to I/O requests initiated from the host. Often, the host computer operating system includes a device driver to communicate with the network adaptor hardware to manage I/O requests to transmit over a network. The host computer further includes a transport protocol driver which packages data to be transmitted over the network into packets, each of which contains a destination address as well as a portion of the data to be transmitted. Data packets received at the network adaptor are often stored in an available allocated packet buffer in the host memory. The transport protocol driver processes the packets received by the network adaptor that are stored in the packet buffer, and accesses any I/O commands or data embedded in the packet.
For instance, the transport protocol driver may implement the Transmission Control Protocol (TCP) and Internet Protocol (IP) to encode and address data for transmission, and to decode and access the payload data in the TCP/IP packets received at the network adaptor. IP specifies the format of packets, also called datagrams, and the addressing scheme. TCP is a higher level protocol which establishes a connection between a destination and a source.
A device driver can utilize significant host processor resources to handle network transmission requests to the network adaptor. One technique to reduce the load on the host processor is the use of a TCP/IP Offload Engine (TOE) in which TCP/IP protocol related operations are implemented in the network adaptor hardware as opposed to the device driver, thereby saving the host processor from having to perform some or all of the TCP/IP protocol related operations. The transport protocol operations include packaging data in a TCP/IP packet with a checksum and other information and sending the packets. These sending operations are performed by a sending agent which may be implemented with a TOE, a network interface card or integrated circuit, a driver, TCP/IP stack, a host processor or a combination of these elements. The transport protocol operations also include receiving a TCP/IP packet from over the network and unpacking the TCP/IP packet to access the payload or data. These receiving operations are performed by a receiving agent which, again, may be implemented with a TOE, a driver, a host processor or a combination of these elements.
FIG. 1 illustrates a stream 10 of TCP/IP packets which are being sent by a sending agent of a source host to a destination host in a TCP connection. In the TCP protocol as specified in the industry accepted TCP RFC (request for comment), each packet is assigned a unique sequence number. As each packet is successfully sent to the destination host, an acknowledgment is sent by the destination host to the source host, notifying the source host by packet sequence number of the successful receipt of that packet. Accordingly, the stream 10 includes a portion 12 of packets which have been both sent and acknowledged as received by the destination host. The stream 10 further includes a portion 14 of packets which have been sent by the source host but have not yet been acknowledged as received by the destination host. The sending agent maintains a TCP Unacknowledged Data Pointer 16 which points to the sequence number of the first unacknowledged sent packet.
The capacity of the packet buffer used to store data packets received at the destination host is generally limited in size. In accordance with the TCP protocol, the destination host advertises how much buffer space it has available by sending a value referred to herein as a TCP Window indicated at 20 in FIG. 1. Accordingly, the source host sending agent uses the TCP Window value to limit the number of outstanding packets sent to the destination host, that is, the number of sent packets for which the source host has not yet received an acknowledgment. A TCP Next Data Pointer 22 points to the sequence number of the next packet to be sent to the destination host. A portion 24 of the datastream 10 between the TCP Next Data Pointer 22 and the end of the TCP Window 20 represents packets which have not yet been sent but are permitted to be sent under the TCP protocol without waiting for any additional acknowledgments because these packets are still within the TCP Window 20 as shown in FIG. 1. A portion 26 of the datastream 10 which is outside the end boundary 28 of the TCP Window 20, is not permitted to be sent under the TCP protocol until additional acknowledgments are received.
As the destination host sends acknowledgments to the source host, the TCP Unacknowledged Data Pointer 16 moves to indicate the acknowledgment of additional packets for that connection. The beginning boundary 30 of the TCP Window 20 shifts with the TCP Unacknowledged Data Pointer 16 so that the TCP Window end boundary 28 also shifts so that additional packets may be sent for the connection. If packets in the portion 14 of the stream 10 remains unacknowledged beyond a certain time interval, the sending agent will typically resend the unacknowledged packets until an acknowledgment is received for those packets.
To initiate the sending of data, the source host typically stores the data of the datastream 10 in a buffer which can be accessed by the sending agent. One such buffer is referred to as a “circular buffer” in which data to be transmitted is copied and is typically kept there until all the data in the circular buffer 10 has been transmitted (and retransmitted as needed) and all of the data has been acknowledged as successfully received.
Circular buffers have been implemented using physical memory which stores data, usually on a short term basis, in integrated circuits, an example of which is a random access memory or RAM. Typically, data can be accessed relatively quickly from such physical memories. A host computer often has additional physical memory such as hard disks and optical disks to store data on a longer term basis. These nonintegrated circuit based memories tend to retrieve data more slowly than the physical memories.
The operating system of a computer typically utilizes a virtual memory space which is often much larger than the memory space of the short term physical memory of the computer. FIG. 2 shows an example of a virtual memory space 50 and a short term physical memory space 52. The memory space of a long term memory such as a hard drive is indicated at 54. The data to be sent in the data stream 10 may initially be stored in noncontiguous portions, that is, nonsequential memory addresses, of the various memory devices. For example, two portions indicated at 10a and 10b may be stored in the physical memory in noncontiguous portions of the physical memory space 52 while another portion indicated at 10c may be stored on the hard drive as shown in FIG. 2. The operating system of the computer uses the virtual memory address space 50 to keep track of the actual locations of the portions 10a, 10b and 10c of the datastream 10. Thus, a portion 50a of the virtual memory address space 50 is mapped to the physical memory addresses of the physical memory space 52 in which the data portion 10a is stored. In a similar fashion, a portion 50b of the virtual memory address space 50 is mapped to the physical memory addresses of the physical memory space 52 in which the data portion 10b is stored. Furthermore, a portion 50c of the virtual memory address space 50 is mapped to the physical memory addresses of the long term hard drive memory space 54 in which the data portion 10c is stored. A blank portion 50d represents an unassigned or unmapped portion of the virtual memory address space 50.
Because memory space in the physical memory is typically relatively limited, portions of the data stored in the physical memory may be copied, that is “swapped” to the hard drive or other long term memory to make room for other data in the physical memory. Thus, the data portion 10c may have been swapped from the physical memory to the hard drive. When data portion 10c was swapped to the hard drive, the virtual memory addresses of the swapped data portion are mapped to the new hard drive physical memory addresses of the hard drive memory space 54 as shown in FIG. 2 instead of the physical memory space 52 at which the portion 10c was previously stored.
FIG. 3 shows prior art operations for sending data in accordance with the Berkeley Software Design (BSD) TCP/IP Stack of the TCP/IP RFC suite. In response to a request to send a datastream 10 of data, the host which may include the computer operating system or driver or both, gathers (block 70) the various data portions 10a, 10b . . . 10n of the datastream 10 from the various locations of physical memory and long term storage. As shown in FIG. 4, the host stores the gathered datastream portions 10a, 10b . . . 10n in contiguous memory locations of the physical memory space 52. In this manner, the data of the datastream 10 is assembled for transmission.
In addition, the host through the host operating system “pins” (block 72) the physical memory locations containing the datastream 10. Pinning these memory locations prevents other applications from causing portions of the data of the datastream 10 from being swapped out to long term memory during the transmission of the datastream 10. The host sends (block 74) the physical memory addresses of the pinned physical memory locations containing the datastream 10 to the sending agent. In response, the sending agent begins sending (block 76) data packets containing the data from the pinned memory locations and begins receiving (block 78) acknowledgments from the destination host as the data packets are successfully sent and received. The sending agent continues sending (block 76) data packets and receiving (block 78) acknowledgments. Unacknowledged packets are resent (block 76) until they are acknowledged (block 78). Once all of the data packets of the data stream 10 have been successfully sent and acknowledged (block 80), the host unpins (block 82) the physical memory locations which contained the data of the datastream 10. Once unpinned, the data stored in the unpinned physical memory locations may be swapped to long term memory as needed.
Notwithstanding, there is a continued need in the art to improve the performance of memory usage in data transmission.